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Phase Diagram of the Triangular t-J Model with Multiple Spin Exchange in the 

Doped-Mott Region 
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We study the triangular t-J model with multiple spin exchange interactions using exact 
diagonalization of small clusters, as a model of monolayer liquid ^He. A phase where the 
energy spectrum suggest the behaviors of spin-charge separation is realized in the doped-Mott 
region due to the competition between ferromagnetic two-spin and antiferromagnetic four-spin 
exchange interactions. This behavior gives an interpretation of the double peaked heat capacity 
recently observed in monolayer liquid '^He adsorbed on a graphite. 
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Doping a Mott insulator opens up a new vista of the 
material physics. which is similar to the evolution of 
the semiconductor physics in the last century. While the 
undoped semiconductor does not have the degree of free- 
dom of either spin and charge, there remains the spin 
degrees of freedom in the Mott insulator. When hole car- 
riers are doped in the Mott insulator, we should handle 
both the spin- and charge-degrees of freedom separately. 
This challenge is, however, not so easy. The problem of 
itinerancy and localization, which is very fundamental 
in quantum physics, are hidden behind this issue. It is 
repeatedly examined, mainly in the context of high-Tc 
cuprates, whether the quasiparticle with spin and charge 
is well defined or not, which is still controversial.^^ The- 
oretically, an interesting spin-charge separation has been 
addressed based on the t-J model in square lattice.^' ^ 
Experimentally, the pseudo (spin) gap state suggests 
the spin-charge separation, but the direct indication of 
spin-charge separation has not been observed yet in the 
photoemission spectroscopy or in the heat capacity.'^' ^ 
The difficulty to measure the spin-charge separation in 
high-Tc cuprates would be due to inhomogeneities or the 
close energy scale of spin and charge. So far the doped 
Mott system has been discussed mainly in the high-Tc 
cuprates. If we can approach the problem from a differ- 
ent standpoint with another system, we can obtain the 
essence of the doped-Mott systems more definitely. 

The second layer of '^He adsorbed on a graphite sur- 
face is another doped-Mott insulator in a purely two- 
dimensional system. In this system, '^He atoms localize 
at a certain density, the so-called 4/7 phase. ^"^^ (We use 
the particle density, n, as ti = 1 at the 4/7 phase.) The 
effective mass exhibits a divergence toward the density 
of the 4/7 phase, n = 1, indicating the Mott transition.^ 
We can, therefore, recognize this ^He system for n < 1 as 
the doped-Mott system. The unique features of this sys- 
tem are: (1) we can control the particle density without 
introducing disorder;^"^ (2) the particles feel the triangu- 
lar lattice potential;^' (3) the base Mott insulator does 
not exhibit any spin order, the gapless spin-liquid,^*^' 
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suggesting the energy scales of spin and charge are well 
separated. 

Recently, a double-peaked structure in the specific 
heat, C, has been reported in the doped-Mott region of 
monolayer "^He.^ For low densities, n < 0.85, C exhibits 
a single broad-peak (or a shoulder) at around Thigh = 50- 
100 mK, and C ocT below Thigh, indicating the Fermi liq- 
uid nature. When n increases, a peak at around Tow =1- 
10 mK appears for n > 0.85 in addition to the reduced 
peak at around Thigh- Then, C shows a double-peaked 
structure for 0.85 < n < 1, namely, in the doped-Mott 
region. It is expected^ that the double-peak structure 
originates from the separation of the energy scale of spin 
and charge, as in one-dimension.^^ (Hereafter we will use 
the term "charge" following the custom of electron sys- 
tem.) There have been, however, no justification of this 
scenario and we have no theories which explain how and 
why such a "spin-charge separation" is realized in this 
system. 

In this letter, we present a theoretical justification of 
the behavior of "spin-charge separation" in the doped- 
Mott region of monolayer liquid '^He. Here, we use the 
word "spin-charge separation" in a broad sense, namely, 
it includes a situation where the energy scale of spin and 
charge is separated leading to a double-peaked structure 
of C, which might be described in the frame of the Fermi 
liquid theory. First an effective model is introduced; the 
t-J model with the multiple spin exchange (MSE) inter- 
action. Then the ground states of this model are inves- 
tigated by the exact diagonalization. We found a phase 
where the energy spectrum suggest the behavior of "spin- 
charge" separation generated by a competition between 
MSEs. 

It is of prime importance to keep in mind the hierarchy 
of the energy scales, when we consider physical problems. 
Although '^He has a strong hard core, there is a process 
in which a '^He atom is excited to the third layer. This 
process corresponds to the effective on-site repulsion U 
of the Hubbard model, which is estimated as J7 ~ 10 K 
from the chemical potential difference between the lay- 
ers. Our interest is below 100 mK, where the whole 
structure of double-peaked C is included. In this case. 
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we can use the t-J model as an effective Hamiltonian as 
discussed in high-Tc cuprates.'^ The hopping energy t is 
estimated from the peak of C to be < ~ Tiiigh ^ 50-100 
mK.9 

It is well known that the MSE is important for hard- 
core quantum systems. For example, for the mono- 
layer solid ^He, J3 ^ J2 ^ Ji was estimated by Bernu et 
al.,"'^^ where J„ is n-spin exchange interaction. Since J3 
term can be transformed into the J2 term as J = J2— 2 J3, 
the sign of the effective two-spin exchange, J, is negative 
near n = l.i8>20 The magnitude of J is estimated to be 
\J\ Tiow ^1-10 mK. The high-temperature expansion 
study yields | J| ^ 3mK, and the four-spin exchange K 
to be positive and K/\J\ ^ 0.2.^" Consequently, the rel- 
evant energy scales below 100 mK are t, J and K which 
satisfy t > \J\ > K. We therefore introduce an effective 
Hamiltonian of the monolayer liquid ^He on the basis of 
the triangular t-J Hamiltonian with the MSE (four-spin) 
as follows: 
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where Si is the spin operator at site i, and P4 is the 
four-spin exchange operators. The projection operator 
Pq = (1 — ni-\nii) excludes the double occupancy, 
which reflects the hard-core potential, so that this t-J-K 
Hamiltonian naturally expresses the hard-core quantum 
systems contrary to the Hubbard model. Near 71 = 1, 
the effective two-spin exchange is ferromagnetic as — ,/ > 
K > 0. At 71 = 1, the t-J-K model is equivalent to 
the previous MSE model {J-K model) for the monolayer 
•^He.^*'^^"^"' (Note that our definition of J is twice as 
large as theirs.) 

It should be noted here that the vacancies in ■^He are 
treated as holes on the triangular lattice in the present 
model. Since the translational symmetry at 4/7 phase is 
different from that of the substrate,^ the translational 
symmetry breaking occurs at a certain density. In or- 
der to consider such possibilities, it will be necessary to 
take account of the interstitial sites in the triangular lat- 
tice. In this paper, we use a simplified model Hamilto- 
nian (1) in order to see whether this minimal model ex- 
hibits "spin-charge separation" . Furthermore, some ex- 
periments^ and recent numerical simulation^^ suggest 
that in the lightly-doped region, the holes move as if it 
keeps the same translational symmetry as that of the 4/7 
phase. Therefore, we think the analysis based on the tri- 
angular lattice will distill the essence of the lightly-doped 
region of monolayer '^He. 

In order to obtain unbiased understandings of this 
model as a first step, we analyze the t-J-K model us- 
ing the exact diagonalization of small clusters. We have 
checked 12-, 18- and 20-site clusters with some cluster- 
shapes and periodic boundary conditions. We obtained 
a phase diagram, whose qualitative property is inde- 
pendent of the cluster size. Hereafter, we set i = 1, 
and mainly show the results for Na = 20 site cluster. 
The phase boundaries are determined from the intersec- 
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Fig. 1. (Color online). Phase diagram of the triangular t-J-K 
model for (a) n = 1 and (b) n = 0.9 with Na = 20. The in- 
set is the phase diagram of the four-spin plaquette. SL, S-C sep., 
and PS stand for spin liquid, possible "spin-eharge separation", 
and phase separation, respectively. 



tions of energy levels, and from the changes of spin- and 
charge-correlation functions, S{q) and N{q). 

At half-filling. First we briefly summarize the phase 
diagram for n = 1. This filling has been studied up to 
36-sitc for K/\J\ ~ and 0.125^'^ with ferromagnetic 
(FM) J, and K/J < 0.125 with antiferromagnetic (AF) 
J.^^ Our results up to 20-site agree well with the previous 
works of 36-site not only qualitatively, but also quanti- 
tatively. Figure 1 (a) shows the phase diagram obtained 
for n= 1. For J > with small K (< 0.04 J), the ground 
state is the three-sublattice Neel ordered state, which is 
verified by the relation^'^'^^ 



E<xSiS+l)/Nx, 



(2) 



where x is the susceptibility per site at zero field. In 
this Neel state, the spin correlation function S{q) ~ 
^a^Hi.jf^"'"^'' {ip-i] - ni\,){ni-[ - n.ii)) shows peaks at 
the corners of the Brillouin zone of triangular lattice, 
the A'-points, indicating the three sublattice structure. 
This three-sublattice Neel state is replaced by a spin liq- 
uid state for 0.04J < K < 0.08J. This phase exhibits 
a large number of singlet excitations in the magnetic 
gap, which is called as the type-II spin liquid (SL-II) by 
Misguich and Lhuillier.^^'^'^ With larger K, the ground 
state changes into a different spin liquid state with a gap. 
This state corresponds to the type-I spin liquid of RVB 
(SL-I).i*'2^ The SL-I phase expands into FM region for 
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Fig. 2. (Color online). Spin correlation function for q^ as a func- 
tion of K for (a) J = 1.0, and (b) J = —0.3. The inset shows the 
contour plot of Xo(<j) a^nd the position of q^. 



X > 0.15| J|. Finally between the FM and the SL-I, there 
is another state, which could correspond to the nematic 
state discussed by Momoi et al.^'' 

Doped-Mott region. Let us now go on to the main sub- 
ject of this Letter, i.e., the doped-Mott region. The phase 
diagram for the doped-Mott region (n = 0.9) is shown 
in Fig. 1 (b). In the region for J > with small K 
(< 0.08 J), the phase-I, the ground state is phase sepa- 
rated, judging from the property of the compressibility 
K = {ma/N^)/{E{N + 2) + E{N - 2) - 2E{N)], where 
E{N) is the ground state energy with N fermions. (As for 
K, we calculated up to Na = 18.) This phase separation is 
similar to those obtained in one-^^ and two-dimensional^^ 
t-J model. We also find that the relation of Eq. (2) holds 
in the phase-I, indicating that the Neel order is formed 
in the phase-separated islands. Furthermore, S{q) shows 
peaks at the iC-points as the three-sublattice Nccl phase 
of n ~ 1. 

The phase separated Neel state is broken by K and 
the ground state changes into the phase-II (and IF). 
The relation of Eq. (2) does not hold any more in the 
phase-II*^ '\ so that the system turns into a homoge- 
neous liquid. The if-dependences of S{q) for various 
in the Brillouin zone arc shown in Fig. 2. The po- 
sitions of are shown in the inset. For J = 1.0 (Fig. 
2 (a)), the peak position, q^, does not change from the 
phase-I to phase-II, but its magnitude becomes smaller. 
From this, we can see that the system possesses strong 
spin-fluctuations of the thrce-sublatticc Nccl order in the 
phase-II near the phase-I. As K increases, however, the 
magnitude of S{q^) approaches to that of S{q^). This 
indicates that the system can be described as a nor- 
mal Fermi liquid state, because the structure of S{q) 
is similar to the non-interacting susceptibility xo{q) ~ 
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Fig. 3. (Color online). Excitation spectrum AB for n = 0.9 with 
J = —0.3. The inset (a) is Ai? in a one-dimensional t-J model 
for n = 0.875 with Na = 16, as a reference of the Tomonaga- 
Luttinger liquid. The inset (b) is Ai? in a triangular t-J model 
for n = 0.33 with Na = 12, as a reference of the Fermi liquid. 



Efe (/fc+q - fk) I {ik - ^k+q) , where fk is the Fermi dis- 
tribution function and is the fermion dispersion. (The 
contour plot of Xo{q) is shown in the inset of Fig. 2.) 
Namely, the spin fluctuation is reduced by increase of 
K. The energy spectrum also indicate the normal Fermi 
liquid property as shown later. The phasc-IF would be- 
long to the phase-II, since the property of S{q) is almost 
the same. Note that there is a possibility that a super- 
fluid is realized in the phase-II', since k is divergently 
large in this region. This situation is similar to the t-J 
model. ^^'^^ If so, the mechanism of the superfluid can be 
due to the spin-fluctuation of three-sublattice Neel state. 

A new phase (phase-Ill) is realized between the phase- 
IV of FM and the phase-II of normal Fermi liquid.^* As 
is shown in Fig. 2 (b) for J = —0.3, the structure of 
S{q.i) in the phase- III is definitely different from those of 
the phase-II and IV. The symmetry of the wave function 
of the phasc-III, therefore, is different from the normal 
Fermi liquid in II and the FM in IV. This is contrary to 
the relation between the phase-I and II. 

The most specific feature of the phase-Ill is its excita- 
tion spectrum. The excitation energy AE of S* = and 
5 = 1 are shown in Fig. 3 for J = —0.3. In the phase-II 
{K > 0.1), AEs=o varies together with AEs=i. This is 
a characteristic property of the typical Fermi liquid (see 
the inset (b) of Fig. 3). In the phase-Ill (0.04 < K < 0.1), 
on the other hand, the energy spectrum behaves differ- 
ently. The valley of AEs=o locate at K ~ 0.1, while that 
of As=i at K ^ 0.04. If the system is a normal Fermi 
liquid, the whole of AE, including AEs=o, will decrease 
toward the FM state as the inset (b) of Fig. 3. However, 
the charge excitation AEs=o in the phase-Ill rather in- 
creases toward the FM state, whereas the spin excitation 
AEs^i keeps decreasing. In this sense, the charge sector 
is separated from the spin sector. This behavior strik- 
ingly resembles that of the Tomonaga-Luttinger liquid, 
which definitely exhibits the spin-charge separation.^' 
(See the inset (a) of Fig. 3.) We speculate that this be- 
havior strongly supports the "spin-charge separation" in 
the phasc-III. From the structures of S{q) and AE, we 
conclude the phase-Ill is a new phase, where spin exci- 
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Fig. 4. (Color online). Illustration of the hole motion between the 
four site plaquette (the shaded region) for (a) the large K region 
and (b) the competing J and K region. 



tations are lowered and charge ones are raised. In such 
a situation, the heat capacity will exhibit the double- 
peaked structure, where the lower peak corresponds to 
the energy scale of the spin excitation and the higher one 
to that of the charge excitation. ^"^ Note that the state in 
the phase III is different from the Tomonaga-Luttinger 
liquid in the respect that the clear sign of spin-charge 
separation in the momentum space cannot be seen. Ac- 
tually, we find no significant peaks both in S(ci) and N(ci) 
(not shown). For the classification whether the phase-Ill 
is the non- Fermi liquid or not, we need further investi- 
gation. 

For obtaining the physical origin of the phase III, it is 
helpful to consider a superlatticc of 4-site.^^ The eigen- 
states of the local four-spin plaquette are as follows: 
E = - J ^ 2K (for S = 0), J, 3J, J - 2K (for S 1), 
5 J -|- 2K (for S ~ 2). Its phase diagram is shown in the 
inset of Fig. 1. In the region of (i), the ground state is 
S* = 0. In (ii), all 5 = 1 states are lower than the others, 
in (iii) 5 = 2 state locates between S' = 1 states, i.e., 
Es=2 is almost degenerate with Es=i, and in (iv) S = 2 
state is the ground state. The qualitative property is very 
similar to the phase diagram of Na = 20, which tells the 
plaquette approximation is effective. 

Let us first consider the large K case. Within the 
plaquette, K favors the S = \ state, i.e., the uuud 
state. In the bulk limit of n = 1, the uuud pla- 
quettes aligns AF with forming the RVB state as 
\uuud,dddu — dddu,uuud). The hole is doped as uuuO, 
not as uudO, since K is no longer valid in the plaque- 
tte. When the hole moves to the neighbor plaquette, the 
previous plaquette becomes uuuu state (Fig. 4 (a)). The 
hole leaves a trace of spin as uuuu, dddd, uuuu..., which 
has an energy loss causing the spinon-holon binding.'^ 
This will be the situation in the phase-II, where the nor- 
mal Fermi liquid is realized by doping in the RVB state. 
Near the FM state, on the other hand, AF K compete 
with FM J, so that the ground states are highly degener- 



ate.' 



In such a situation, the uuuu is the alternative 



to uuud in the plaquette. Even when the hole moves to 
the neighbor plaquette, the previous plaquette keeps the 
degenerate state of uuud and uuuu (Fig. 4 (b)), namely, 
the hole does not leave a trace of energy-loss configu- 
rations. This is the very situation of the "spin-charge 
separation" . The spin excitation comes from the excita- 
tion between uuud and uuuu state, while the charge one 



from the motion of holes. Therefore, the key to the "spin- 
charge separation" is the degeneracy of uuud and uuuu 
state (the competing J and K). In other words, doping a 
gapless spin-liquid produces the "spin-charge separated" 
liquid. 

In this Letter, the t-J-K model was introduced as an 
effective model of the monolayer liquid '^He. A new phase 
(phase-Ill) was found near the FM phase in the doped- 
Mott region. The competition between the FM J and 
AF K generates the phase of possible "spin-charge sepa- 
ration" . The triangular t-J-K model therefore naturally 
unravel the mysterious double-peaked heat capacity in 
monolayer liquid ■^He. Our results suggest that doping a 
gapless spin-liquid yields a spin-charge separated state. 
These results open up a new possibility of the doped- 
Mott systems. 
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